Optimizing management of autonomous vehicles

ABSTRACT

A system, device and method of managing autonomous vehicles are provided. The system may include a server, a feedback device, a control device for controlling a vehicle. A server may include a communication interface configured to communicate with a control device of each of a plurality of vehicles and a feedback device; a memory storing instructions; and at least one processor. The at least one processor is configured to receive control data from the control device of each of the plurality of vehicles; determine an operation status of each of the plurality of vehicles based on the control data; generate a feedback interface based on the operation status of each of the plurality of vehicles; and transmit the feedback interface to the feedback device.

BACKGROUND 1. Technical Field

The disclosure relates to the field of autonomous driving technology,and in particular, to a system and a method of managing autonomousvehicles in a parking space or a distribution center with a safetyback-up.

2. Description of Related Art

With the continuous development of autonomous vehicles, a framework foroptimizing and managing autonomous vehicles is also evolving. Whileautonomous vehicles may be managed in a distributed manner, that is,each vehicle controlling its own affairs independent of another vehicle,with the deployment of numerous autonomous vehicles in certain places,it is necessary to have a centralized management system for overseeingand controlling operations of each vehicle. Such a centralizedmanagement system will allow vehicles to be effectively and safelymanaged in a parking lot or a distribution center.

The existing parking controllers are configured such that instructionsfor controlling the vehicles are automatically executed even when theseinstructions may not be accurate under certain circumstances. Due tosuch an automatic execution of inaccurate instructions, the vehicle andother objects may be damaged, and may pose a safety risk for people. Inaddition, the existing parking controllers lack a user-friendlyinterface in which a driver can reject centralized control commands froma server, or an interface in which a third person (e.g., a safetyoperator) can intervene in an automatic execution of commands to promotesafe operation of the autonomous vehicles.

Therefore, there is a need for a centralized management system thatefficiently and safely manages autonomous vehicles in parking spaces anddistribution centers, and that provides user-friendly interfaces toexecute or reject control commands to the vehicles.

SUMMARY

The disclosure provides a system and a method of managing autonomousvehicles in parking spaces and distribution centers, and for providinguser-friendly interfaces to execute or reject control commands of thevehicles.

In accordance with an aspect of the disclosure, there is provided afeedback device including: a display; a communication interfaceconfigured to communicate with a server and a control device of each ofa plurality of vehicles; and a processor configured to: receive afeedback interface from the server, control the display to display thefeedback interface; receive a user input on the feedback interface; andtransmit a control command corresponding to the user input to thecontrol device of each of the plurality of vehicles.

The feedback interface includes an image of at least one vehicle amongthe plurality of vehicles.

The feedback interface further includes images of one or more componentsof the at least one vehicle.

The feedback interface further includes an operation status of the oneor more components of the at least one vehicle.

The feedback interface further includes a metric for the one or morecomponents of the at least one vehicle.

The image of the at least one vehicle is a top-view of the at least onevehicle.

The feedback interface activates or deactivates one or more interfacesbased on a heartbeat signal indicating an operating status of the one ormore components of the at least one vehicle.

The feedback interface activates or deactivates one or more interfacesbased on the control command being executed by the control device ofeach of the plurality of vehicles.

The feedback interface configures one or more interfaces to blink basedon a latency of transmitting the control command.

The feedback interface further includes a map view of the plurality ofvehicles.

In accordance with an aspect of the disclosure, there is provided amethod of controlling a feedback device connected to a server. Themethod includes: receiving a feedback interface from the server;controlling a display to display the feedback interface; receiving auser input on the feedback interface; and transmitting a control commandcorresponding to the user input to a control device of each of aplurality of vehicles.

The feedback interface includes images of one or more components of theat least one vehicle.

The feedback interface further includes an operation status of the oneor more components of the at least one vehicle.

The feedback interface activates or deactivates one or more interfacesbased on a heartbeat signal indicating an operating status of the one ormore components of the at least one vehicle.

The feedback interface activates or deactivates one or more interfacesbased on the control command being executed by the control device ofeach of the plurality of vehicles.

The feedback interface configures one or more interfaces to blink basedon a latency of transmitting the control command.

In accordance with an aspect of the disclosure, there is provided aserver including: a communication interface configured to communicatewith a control device of each of a plurality of vehicles and a feedbackdevice; a memory storing instructions; and at least one processor. Theat least one processor is configured to: receive control data from thecontrol device of each of the plurality of vehicles; determine anoperation status of each of the plurality of vehicles based on thecontrol data; generate a feedback interface based on the operationstatus of each of the plurality of vehicles; and transmit the feedbackinterface to the feedback device.

The at least one processor is configured to execute: a priority moduleconfigured to determine a priority of each of the plurality of vehiclesto be moved; a path minimization module configured to minimize a pathdistance for each of the plurality of vehicles from a first location toa second location; and a time scheduling module configured to obtain aschedule delivery time for each of the plurality of vehicles anddetermine a schedule time to be deployed for each of the plurality ofvehicles.

The control data includes at least one of vehicle data, positioning dataor sensor data.

The priority module is further configured to assign the priority of theplurality of vehicles based on at least one of an urgency of an itembeing carried in a vehicle, a duration of the item carried in thevehicle, or a maximum time for delivery assigned to the item or apassenger in the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects, features, and advantages of certainembodiments of the disclosure will be apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a diagram illustrating a central management system formanaging autonomous vehicles according to an embodiment;

FIG. 2 is a diagram illustrating a server of a central management systemaccording to an embodiment;

FIG. 3 is a flowchart illustrating a method of providing feedbackinterface to a feedback device by a server according to an embodiment;

FIG. 4 is a diagram illustrating a feedback device according to anembodiment;

FIG. 5 is a flowchart illustrating a method of performing a controlcommand on an autonomous vehicle based on a user input to a feedbackinterface according to an embodiment;

FIG. 6 is a schematic diagram illustrating an exemplary feedbackinterface according to an embodiment:

FIG. 7 is a diagram illustrating a control device according to anembodiment;

FIG. 8 is a flowchart illustrating a method performing a control commandaccording to an embodiment;

FIG. 9 is a flowchart illustrating a method of performing a controlcommand according to another embodiment; and

FIG. 10 is a schematic diagram illustrating an exemplary controlinterface according to an embodiment.

DETAILED DESCRIPTION

Embodiments of the disclosure will be described in detail with referenceto the accompanying drawings. The same reference numerals used in thedrawings may identify the same or similar elements. The terms used inthe disclosure should not be strictly construed as defined in thedisclosure, but should be construed as those one of ordinary skilled inthe art would understand in the context of the disclosure. Hereinafter,embodiments of the disclosure will be described with reference to theaccompanying drawings. However, it should be noted that the embodimentsof the disclosure may be in different forms and are not limited to theembodiments of the disclosure set forth herein.

FIG. 1 is a diagram illustrating a central management system formanaging autonomous vehicles according to an embodiment.

Referring to FIG. 1 , a central management system 10 for managingautonomous vehicles may include a server 100, a feedback device 200, acontrol device 300, a network 190 and a plurality of autonomous vehicles390. The server 100, the feedback device 200 and the control device 300may be connected to each other through the network 190. The controldevice 300 may be implemented in each of the plurality of autonomousvehicles 390. The feedback device 200 and the control device 300 may beany computing device, including a desktop computer, a laptop, a workstation, a mobile computing device, etc. The server 100 may be a cloudserver, and may be implemented as an independent server or as a servercluster including a plurality of servers.

The network 190 may include wired and/or wireless communicationnetworks, and may include any number of networks. The network 190 mayexchange data in circuit-switched and/or packet-switched channels. Forexample, the networks 190 may include telecommunication networks, localarea networks, wide area networks, WiFi networks, and/or the Internet.

FIG. 2 is a block diagram illustrating a server for controllingautonomous vehicles according to an embodiment.

The server 100 may include a processor 110, a memory 120, acommunication interface 130, an input/output interface 140, and astorage 160. The feedback device 200 (shown in FIG. 4 ) may include aprocessor 210, a memory 220, a communication interface 230, aninput/output interface 240, a display 250, and a storage 260. Thecontrol device 300 (shown in FIG. 6 ) may include a processor 310, amemory 320, a communication interface 330, an input/output interface340, a display 360 and a storage 360. As set forth above, the componentsor elements of the server 100, the feedback device 200, and the controldevice 300) may be configured to perform the same or similar functions.Therefore, for the convenience of description, the components of theserver 100 will be mainly described herein.

Referring to FIG. 2 , the processor 110 may control an overall operationof the server 100. Specifically, the processor 110 may be connected toand configured to control the operations of the memory 120, thecommunication interface 130, the input/output interface 140 and thestorage 160. The processor 110 may be implemented according to variousembodiments. For example, the processor 110 may be implemented as atleast one of an application specific integrated circuit (ASIC), anembedded processor, a microprocessor, hardware control logic, a hardwarefinite state machine (FSM), a digital signal processor (DSP), a neuralnetwork processor (NPU), or the like. The processor 110 may include acentral processing unit (CPU), a graphic processing unit (GPU), and amain processing unit (MPU), or the like. In addition, the processor 110may include one or more processors.

The memory 120 may store at least one instruction and various softwareprograms or applications for operating the server 100 according to theembodiments of the disclosure. For example, the memory 120 may include asemiconductor memory, such as a flash memory, a magnetic storage mediumsuch as a hard disk, or the like. The memory 120 may refer to anyvolatile or non-volatile memory, a read-only memory (ROM), a randomaccess memory (RAM) communicatively coupled to the processor 110 or amemory card (e.g., a micro SD card, a memory stick) connectable to theserver 100. The memory 120 may store various software modules or codesfor operating the server 100, and the processor 110 may control theoperations of the server 100 by executing various software modules thatare stored in the memory 120. That is, the memory 120 may be accessed bythe processor 110 to perform data reading, recording, modifying,deleting, updating or the like. Further, the memory 120 may storeexecutable instructions, code, data objects etc.

The communication interface 130 may include circuitry or an interfacethat is configured to communicate with an external device (e.g., afeedback device 200 or a control device 300) through the network 190.The communication interface 130 may include at least one of a Wi-Fimodule, a Bluetooth module, a wireless communication module, or a nearfield communication (NFC) module. Specifically, the Wi-Fi module maycommunicate by a Wi-Fi method and the Bluetooth module may communicateby a Bluetooth method. When using the Wi-Fi module or the Bluetoothmodule, various connection information such as service set identifier(SSID) may be transmitted and received for communication connection andthen various information may be transmitted and received through thecommunication interface 130.

The input/output interface 140 may be configured to receive an inputfrom a user or other devices, and the processor 110 may receive a usercommand for controlling the operations of the server 100 through theinput/output interface 140. The input/output interface 140 may include,for example, a microphone, a camera, a remote controller, a keyboard, amouse, or the like.

The storage 160 may be configured to store various program, application,instructions, modules, code, models, data object, etc. for computingvarious data related to managing autonomous vehicles. Here, the storage160 may be provided in the form of a non-transitory storage medium thatis a tangible device that may not include a signal (e.g.,electromagnetic wave). In particular, the storage 160 may store apriority module 162, a path minimization module 164 and a timescheduling module 166. However, one or more embodiments of thedisclosure are not limited thereto, and the storage 160 may include anyother module that is relevant to managing autonomous vehicles.Furthermore, these modules may be temporarily stored in the memory 120to be processed by the processor 110.

The priority module 162 may be configured to determine a priority ofeach of a plurality of autonomous vehicles 390 and reorder the vehiclemovement in a parking space or a distribution yard. Here, each vehiclemay be assigned a vehicle ID, and the priority may be linked to thevehicle ID. By way of example, the priority may be assigned to eachvehicle based on urgency of an item being carried in a vehicle, aduration of an item carried in the vehicle and/or a maximum time fordelivery assigned to an item or a passenger (e.g., an emergencypatient). Based on the weights and predetermined thresholds assigned toeach condition, one vehicle may be prioritized over another vehicle tobe moved from one location to another. When a vehicle satisfies apredetermined threshold assigned to a certain condition, an urgency flagmay be activated for the vehicle to be prioritized over other vehiclesin the parking lot or the distribution center.

The path minimization module 164 may be configured to compute a distancebetween a start point to a target destination of the vehicle andminimize a path distance of the vehicle. For example, the pathminimization module 164 may compute a plurality of available routesbetween the start point and the target destination of the vehicle andselect one of the plurality of routes that minimizes the path distancebetween the start point and the target destination. The priority of eachvehicle may be considered in terms of calculating a minimum distance foreach vehicle. For example, the path minimization module 164 may beoptimized by selecting a shorter path for a higher priority vehicle.Specifically, in an equation for calculating a minimum path distance ofeach vehicle, a priority assigned to each vehicle may be used asmultipliers for parameters in the equation. In addition, an energythreshold flag may be implemented in the same equation described aboveand may be used as an exponential multiplier for vehicles with lowerpriorities.

The time scheduling module 166 may be configured to schedule a deliverytime of each vehicle. For example, the time scheduling module 166 mayobtain a scheduled delivery time of an object (e.g., an item or apassenger) being carried in each vehicle and schedule a time for eachvehicle to be deployed. Here, the time scheduling for delivery mayintegrate concepts such as “Just in Time” (JIT) to improve manufacturingefficiency, and to reduce a storage space and inventory amortizationcosts.

While some example modules are described above, one or more embodimentsof the disclosure are not limited thereto. The server 100 may beconfigured to implement other modules in order to optimize themanagement of autonomous vehicles. For example, one or more additionalmodules may be configured to extract geometry data of a parking spacewhen a vehicle enters the parking space, and the extracted geometry datamay be used to optimize a spatial usage in which all the vehicles may befitted in a limited space of a parking lot. Furthermore, the one or moreadditional modules may be configured to implement an order of retrievalconcept in which the priority and urgency values can be accounted for incalculating a time for retrieval, and the vehicles may be retrievedaccording to the calculated time for retrieval. Also, in addition toadopting First-In First-Out (FIFO) for managing autonomous vehicles in aparking space, a unique revolving priority number may be assigned toeach vehicle so as to determine a specific retrieval order for eachvehicle.

Furthermore, the server 100 may include one or more additional modulesthat may be configured to compute various solutions considering factors,such as latency and a vehicle emergency. Here, the term “solution” maymean a next course of action to be taken by certain part of component ofa vehicle. Latency may be occur as solutions for vehicle actions arebeing computed in the server 100 (e.g., cloud server), and as thecomputed solutions are being transmitted to the respective vehicles.Therefore, latency may be taken into consideration by the cloud serverwhen computing solutions for the vehicles. Specifically, the latency maybe calculated by comparing timestamps of actions performed by thevehicles. That is, one or more vehicles 390 may transmit a vehicleaction (e.g., moving a predetermined distance, stopping at a stop sign,etc.) with a timestamp to the server 100. The server 100 may receiveeach vehicle action with a timestamp through a methodology such as GPS.Based on the received timestamps, if the calculated latency is greaterthan or equal to a predetermined threshold (e.g., set by parameters ofthe server), then a solution may be recalculated in the server 100.Here, for example, the calculated latency being greater than or equal tothe predetermined threshold may mean that sensor data received from avehicle is old or outdated, and the actual position of the vehicle mayhave changed during the period of latency, thereby making the previoussolution calculated for the vehicle to be infeasible. Therefore, arecalculation is performed by the server 100 based on the latestavailable values of vehicle actions transmitted from the vehicle, andthen the recalculated solution (or an updated solution) is transmittedto the vehicle.

In addition, the vehicles may include on-board vehicle sensors that maybe used as an emergency response resource in a case where the vehicle'sdetection units fail to or are unable to account for close rangeobstacles. In such a case, the server 1N may receive an emergencydetection based on the on-board vehicle sensors and a solution may bepromptly calculated and transmitted back to the vehicle.

As described above, it is understood that the above-described modulesmay be distributed across a plurality of servers.

In addition, according to various embodiments of the disclosure, methodsand devices disclosed herein may be provided as software of a computerprogram product. A computer program product may be distributed in theform of a machine readable storage medium (e.g., compact disc read onlymemory (CD-ROM)) or distributed online through an application store orbetween two devices directly. In the case of on-line distribution, atleast a portion of the computer program product (e.g., a downloadableapp) may be stored temporarily or at least temporarily in a storagemedium such as a manufacturer's server, a server in an applicationstore, or a memory in a relay server.

Although some example components of the server 100 are described above,it is understood that one or more embodiments of the server 100 are notlimited thereto, and the server 100 may include more or less componentsdepending on the need for managing autonomous vehicles.

FIG. 3 is a flowchart illustrating a method of providing a feedbackinterface to a feedback device by a server according to an embodiment.

Referring to FIG. 3 , in step S302, the server 100 may receive controldata from the control device 300 of the autonomous vehicle 390. Thecontrol data may include vehicle data, positioning data, sensor data,and other data related to the functioning of the vehicle. The vehicledata may include attributes of a vehicle, such as a speed, a tirepressure, a fuel level, an engine status, a battery status, a steeringangle, a door status, etc. However, the vehicle data described above isnot limited thereto, and the vehicle data may include any otherattributes that relate to the functioning of a vehicle. The positioningdata may include location-based data, such as global positioning system(GPS) data. However, the positioning data described above is not limitedthereto, and may include any other data that relates to identifyinglocations of the vehicle and other objects. The sensor data may includedata for tracking and classifying objects based on sensors, such as aLiDAR, a radar and/or a camera. While some example control data aredescribed above, the server 100 may receive any other types of data fromthe control device 300 that may be relevant to ensuring efficient andsafety operations of the autonomous vehicles.

In step S304, the server 100 may determine an operation status of thevehicle based on the received control data. By way of example, anoperation of opening a door of the vehicle may be received in the formof the vehicle data from the control device 300.

In step S306, the server may generate a feedback interface based on thedetermined operation status of the vehicle. The feedback interface is aninterface designed to guarantee safe operation of a vehicle and anefficient handling of control commands of a vehicle. For example, thefeedback interface may include one or more interactive interfacescorresponding to respective control commands to a vehicle. The feedbackinterface will be described in more detail below with reference to FIGS.4-6 .

In step S308, the server 100 may transmit the feedback interface to thefeedback device 200 (shown in FIG. 1 ), to be displayed to a user (e.g.,a safety operator) of the feedback interface. According to anembodiment, the feedback interface may be only activated when anautomation of a control command stops working. According to anotherembodiment, the feedback interface may be activated when a user of thefeedback interface decides to interfere with certain operations of avehicle. The user of the feedback interface may manually and remotelycontrol certain operations of a vehicle.

FIG. 4 is a diagram illustrating a feedback device according to anembodiment.

As set forth above, the components or elements of the server 100, thefeedback device 200, and the control device 300 may be configured toperform the same or similar functions. Therefore, overlappingdescriptions thereof may be omitted.

Referring to FIG. 4 , the feedback device 200 may include a processor210, a memory 220, a communication interface 230, an input/outputinterface 240, a display 25) and a storage 260. The display 250 may beimplemented as a liquid crystal display (LCD) panel, organic lightemitting diodes (OLED), a flexible display, a touch screen display, atransparent display, or the like. The processor 210 may control thedisplay 250 to display image signals received from the memory 220 of thefeedback device 20 or received from an external device (e.g., the server100) through the communication interface 230. In addition, the display350 of the control device 300 (shown in FIG. 7 ) may be configured toperform the same or similar functions as the display 250 of the feedbackdevice 200. However, the implementations of the displays 250 and 350 arenot limited thereto. The feedback device 200 will be described in moredetail with reference to FIG. 5 herein below.

FIG. 5 is a flowchart illustrating a method of performing a controlcommand on an autonomous vehicle based on a user input to a feedbackinterface according to an embodiment.

Referring to FIG. 5 , in step S502, the processor 210 may receive afeedback interface from the server 100 through the communicationinterface 230.

In step S504, the processor 210 may control the display 250 to displaythe feedback interface received from the server 100. The feedbackinterface may include interfaces corresponding to a current operation ora metric of a vehicle. Specifically, on the feedback interface, a visualrepresentation of a vehicle and any metric or operation status of thecomponents or parts of the vehicle may be presented. For example, thefeedback interface may include a current status of a tire pressure asbeing 25 Psi. However, one or more embodiments of the disclosure are notlimited thereto, and the feedback interface may include other operationsor metrics of a vehicle, such as doors/windows, an engine, and any othervehicle/automotive parameters. An exemplary feedback interface will bedescribed in more detail with reference to FIG. 6 herein below.

In step S506, the processor 210 may receive a user input on the feedbackinterface through the input/output interface 240. Here, a user (e.g., aperson responsible for monitoring and controlling management ofautonomous vehicles) who is viewing the feedback interface may input acontrol command on the feedback interface to perform the control commandon a vehicle. For example, the user may input a control command to stopthe vehicle based on learning from one of the metrics displayed on thefeedback interface that the fuel level is low.

In step S508, the processor 210 may transmit a control command input bya user to the server 100 through the communication interface 230.Alternatively, the processor 210 may transmit a control command directlyto an autonomous vehicle through the communication interface 230.

FIG. 6 is a schematic diagram illustrating an exemplary feedbackinterface according to an embodiment.

Referring to FIG. 6 , the feedback interface may display a top-view of avehicle including certain status of an operation or a metric of thevehicle. By way of examples, a tire pressure metric of “25 Psi” may bedisplayed next to a tire from the top-view of the vehicle to indicatethe status of the tire, a GPS status may be displayed on top of thetop-view of the vehicle to indicate whether the GPS of the vehicle isfunctioning properly (e.g., “OK”), and an opened door may be displayedon the top-view of the vehicle to indicate that the door of the vehicleis currently open. However, the layout of the feedback interface is notlimited to the above examples. For example, a top-view of the passengersin each seat of the vehicle may be displayed based on a detection of thepassengers by an in-vehicle monitoring camera.

In addition, the top-view layout may not only represent a singleautonomous vehicle, but may display a plurality of autonomous vehiclesin a map view. Here, each vehicle among the plurality of autonomousvehicles may be designated different colors, arrows and patterns toindicate controlled or uncontrolled vehicles. Also, any available routesmay be displayed on the top-view of a map, and certain sections of theroutes may be highlighted and/or selected to indicate a position of thevehicle on the map and/or routes to be taken by the vehicle. Theselection of the routes may be made by simply tapping on a section ofthe route in a case where the display 250 is a touch screen display,however, a method of receiving a user input by the display 250 is notlimited thereto.

Furthermore, a user (e.g., a safety operator) may actively interact withthe feedback interface to perform certain operations on the vehicles.The feedback interface may be implemented on a touch screen display.However, one or more embodiments are not limited thereto, and the usermay interact with the feedback interface through various other means,such as voice recognition.

As described above, the feedback interface may display a featured maparound the vehicle. For example, the feature map may include a parkinglot, and the user of the feedback interface may select one of the parkedvehicles in the parking lot with a touch on a touch screen display andguide the selected vehicle to be parked at another spot by touching anempty parking spot within the featured map. In addition, the featuredmap may also display overlays of different routes that may be taken bythe selected vehicle and the user may select one of the routes to guidethe selected vehicle to travel along the selected route. As anotherexample, the user may select multiple vehicles operating as fleets ordelivery vehicles and guide those vehicles to a certain destination (orsummon) point shown on the map to pick-up or drop-off an item.

Furthermore, the feedback interface may include other visual highlightsbased on an execution of a control command and may be based on aheartbeat signal from the vehicle. Here, the term “heartbeat signal” mayrefer to a signal transmitted from a vehicle indicating that respectivefunction associated with each component or part of the vehicle isoperating properly. The heartbeat signal may be obtained by performingperiodic sampling on each component or part of the vehicle to determinewhether certain component or part of the vehicle has stoppedfunctioning. By way of example, if a heartbeat signal is not received bythe server 100 from the control device 300 in one of the vehicles 390,one or more user interfaces (e.g., a button) in the feedback interfacemay be “greyed” out or deactivated to indicate that a control commandrelating to certain components or parts of the vehicle cannot beperformed. For example, if a certain control command (e.g., a command tochange lane) would lead the vehicle to enter an area where a detectionunit or a sensor of the vehicle may become unavailable or disabled, sucha control command may not be accessible (e.g., “greyed out” on thefeedback interface) to the user of the feedback interface so as toprevent the detection unit or the sensor of the vehicle becomingdisabled. This feature may be referred to as “Command Denial at Source.”

According to an embodiment, when one or more control commands are beingrejected by the control device 300 of the vehicle 390, which willdescribed in detail below with reference to FIG. 10 , the feedbackinterface may grey out one or more interfaces that correspond to the oneor more control commands being rejected by a user of the control device300. As another example, when the vehicle is currently executing acontrol command, the feedback interface may change a color of aninterface (e.g., a button) corresponding to the control command, to turnyellow, where a yellow button would indicate the control command iscurrently being executed in the vehicle.

Furthermore, the feedback interface may display one or more interfacestaking into account for latency of issuing one or more control commands.That is, a user of the feedback interface may be informed of the latencyvia sequence of blinks of an interface to indicate the latency of thecontrol command. Here, the sequence of blinks on the interfacecorresponding to the control command may eventually turn to a greencolor to indicate the operational status of the control command on thevehicle. However, for example, if there is an obstacle that hinders thecompletion of the control command, then the interface may turn red toindicate that the control command is still in a queue of the memory 220waiting to be executed.

Although some examples of the feedback interface are described above,one or more embodiments of the feedback interface are not limitedthereto.

FIG. 7 is a diagram illustrating a control device according to anembodiment.

As described, the components or elements of the server 100, the feedbackdevice 20X), and the control device 300 may be configured to perform thesame or similar functions. Therefore, overlapping descriptions thereofmay be omitted herein below.

Referring to FIG. 7 , the control device 300 may include a processor310, a memory 320, a communication interface 330, an input/outputinterface 340, a display 350 and a storage 360. The storage 360 mayinclude control data such as vehicle data 362, positioning data 364 andsensor data 366. The vehicle 362 data may include data relating toattributes of a vehicle, such as a speed, a tire pressure, a fuel level,an engine status, a battery status, a steering angle, a door status,etc. The positioning data 364 may include location-based data, such asglobal positioning system (GPS) data. The sensor data 366 may includedata for tracking and classifying objects based on sensors, such as aLiDAR, a radar and a camera. While some examples of control data aredescribed above, one or more embodiments are not limited thereto, andthe storage 360 may store any other data that may be relevant toensuring efficient and safety operations of the autonomous vehicles.

The control device 300 may be implemented in each vehicle among theplurality of vehicles 391 (shown in FIG. 1 ). Alternatively oradditionally, the control device 3M) may be separated from the vehicle.For example, the control device 3M) may be implemented on a terminaldevice (e.g., a cell phone) that may be configured to control operationsof the vehicle.

FIG. 8 is a flowchart illustrating a method performing a control commandaccording to an embodiment.

Referring to FIG. 8 , in step S802, the processor 310 may transmitcontrol data (e.g., vehicle data, positional data, and sensor data) tothe server 100 through the communication interface 330. However, one ormore embodiments are not limited thereto, and the processor 310 maytransmit any other data to the server 100 to ensure efficient and safeoperations of the autonomous vehicles.

In step S804, the processor 310 may receive at least one control commandfrom the server 100 through the communication interface 330. Here, atleast one control command may be generated based on computationsperformed by various modules (e.g., the priority module 162, the pathminimization module 164, the time scheduling module 166) included theserver 100, and may instruct certain operations to be automaticallyperformed on the vehicle. However, the embodiment is not limitedthereto, and the processor 310 may also receive the at least one controlcommand directly from the feedback device 200 through the communicationinterface 330.

In step S806, the processor 310 of the control device 300 may instructvarious components or parts of the vehicle to perform the receivedcontrol command.

FIG. 9 is a flowchart illustrating a method of performing a controlcommand according to another embodiment.

Referring to FIG. 9 , in step S902, the processor 310 of the controldevice 300 may receive at least one control command from at least one ofthe server 100 or the feedback device 200.

In step S904, the processor 310 may control the display 350 to displaythe received control command on a control interface. Here, the controlinterface may also include other executable buttons or interfaces. Forexample, the control interface may include an in-built rejectioninterface (shown in FIG. 10 ), so as to receive a user input withrespect to the displayed control command. The in-built rejection systemwill be described in more detail below with reference to FIG. 10 .However, one or more embodiments are not limited thereto, and thecontrol interface may include any other interfaces that may be relevantto ensuring efficient and safe operations of the autonomous vehicles.

In step S906, the processor 310 may receive a user input on the controlinterface through the input/output interface 340.

In step S908, the processor 310 may determine whether the received userinput on the control interface is rejecting the received controlcommand. If the user input rejects the received control command (stepS908: Yes), then the control command is not performed on the vehicle.However, if the user input does not reject the control command (stepS908: No), then the control command may be automatically performed onthe vehicle. However, one or more embodiments are not limited thereto.For example, in a case where the user is inactive, and the controlinterface does not receive any user input from the user for apredetermined time period, then the control command may be automaticallyexecuted on the vehicle. Alternatively, in the same scenario asdescribed above, when the control interface does not receive any userinput for the predetermined time period, then the control command maynot be executed.

FIG. 10 is a schematic diagram illustrating an exemplary controlinterface according to an embodiment.

Referring to FIG. 10 , the control device 300 may include an in-builtrejection system in which a user of the control device 300 may reject acontrol command issued by the feedback device 200 or the server 100.Here, the processor 310 of the control device 300 may control thedisplay 350 to display the control interface shown in FIG. 10 . Thecontrol interface may include an interface 1002 displaying the issuedcontrol command by the feedback device 200 or the server 100 and aninterface 1004 for receiving an input from the user of the controldevice 300 to reject the control command. By way of example, when theissued control command is to travel 10 m backward, the user of thecontrol device 300 may stop the automatic execution of such controlcommand by touching or pressing the “stop” interface 1004 to stop theautomatic execution of the control command of travelling 10 m backward.Here, the user may be provided with this in-built rejection system tocontrol any inaccurate automatic execution of a control command in whicha pedestrian or another object may be at risk due to such controlcommand. However, one or more embodiments are not limited thereto, andthe control interface may include various interfaces to provide in-builtrejection interface. For example, the in-built rejection system may alsointeract with a user through an audio signal in which the user canreject control commands by way of speaking to a microphone.

Furthermore, the in-built rejection system may be applied to one or morecommands. The in-built rejection system may detect multiple similarcommands received within a predetermined period. For example, thein-built system may detect multiple similar commands within apredetermined period of ‘x’, where ‘x’ is directly proportional to afactor that accounts for both an acceleration and a velocity of avehicle. This relationship may be expressed in Equation (1) as follows:X=Av+Ba  (1)

Here, “v” is a velocity of a vehicle, “a” is a lateral acceleration ofthe vehicle, and “A” and “B” are fixed constants.

According to an embodiment, when multiple distinct and directlycontradicting commands may be received in the period of “x,” thein-built rejection system of the control device 300 may be configured toautomatically reject at least one of the multiple commands. For example,if a first control command and a second control command are receivedwith the period “x,” the second control command may be automaticallyrejected, where the second control command is received after the firstcontrol command.

By way of example, when the control device 300 receives a first controlcommand to travel straight and a second control command to change lane,the second control command may be automatically rejected.

According to another embodiment, the in-built rejection system may takeinto consideration the vicinity of a vehicle. The vicinity of a vehiclemay include one or more objects or obstacles (e.g., a pedestrian oranother vehicle), and may be determined based on a distance from thevehicle to the one or more other objects or obstacles and a relativevelocity of the vehicle with respect to the one or more other objects.The vicinity of a vehicle may be calculated according to Equation (2) asfollows:V=Cv+Dd  (2)

Here, “v” is a relative velocity, “d” is a distance from the vehicle toanother object, and “C” and “D” are fixed constants.

By way of example, in the same case described above, when the controldevice 300 receives the first control command to travel straight and thesecond control command to change lane, the in-built rejection system maycalculate the vicinity of the vehicle and determine whether there areany other objects in the vicinity of the vehicle. Based on determiningthat there is an object in the vicinity, the in-built rejection systemmay reject the second control command to change lane because executingthe second control command may cause a collision between the vehicle andthe detected object.

Although some example operations of the in-built rejection system aredescribed above, one or more embodiments are not limited thereto. Forexample, control commands may be rejected based on vehicle dynamicsfeasibility. That is, when control commands are issued to abruptlychange a direction of a vehicle, perform a lane change at a higher speedor perform an action that is not feasible according to a loadcharacteristic of a semi-truck vehicle, such control commands may beinfeasible and thus, may be rejected by the in-built rejection system ofthe control device 300. Some exemplary embodiments of the in-builtrejection system are described above. However, the in-built rejectionsystem is not limited thereto, and certain control commands may berejected by the in-built rejection system according to a geo-fencingtechnology.

The exemplary embodiments of the disclosure have been shown anddescribed above, however, the embodiments of the disclosure are notlimited to the aforementioned specific embodiments. It may be understoodthat various modifications, substitutions, and improvements can be madeby those having ordinary skill in the art in the technical field towhich the disclosure belongs, without departing from the spirit andscope of the disclosure as claimed by the appended claims. It should beunderstood that such modifications, substitutions, and improvementsshall fall within the protection scope of the disclosure, and should notto be construed independently from the technical idea or prospect of thedisclosure.

What is claimed is:
 1. A central management system for managingautonomous vehicles comprising a feedback device and a control device,wherein the feedback device comprises: a display; a communicationinterface configured to communicate with a server and the control deviceof each of a plurality of vehicles; and a processor configured to:receive a feedback interface from the server; control the display todisplay the feedback interface; receive a user input on the feedbackinterface; and transmit a control command corresponding to the userinput to the control device of each of the plurality of vehicles, andwherein, when the control device receives a first control command and asecond control command within a preset time duration and the secondcontrol command contradicts the first control command, the controldevice is configured to automatically reject the second control command.2. The central management system of claim 1, wherein the control deviceis configured to automatically reject the second control command that isreceived after the first control command within the preset timeduration.
 3. The central management system of claim 1, wherein, when thefirst control command and the second control command that are receivedwithin the preset time duration contradict each other, the controldevice is configured to calculate a vicinity of each of the plurality ofvehicles, determine whether there is any object in the vicinity of eachof the plurality of vehicles, and automatically reject the secondcontrol command which, when executed, causes a collision between atleast one vehicle of the plurality of vehicles and the object.
 4. Thecentral management system of claim 3, wherein the feedback interfaceactivates or deactivates one or more interfaces based on a heartbeatsignal indicating an operating status of one or more components of atleast one vehicle among the plurality of vehicles.
 5. The centralmanagement system of claim 1, wherein the feedback interface furthercomprises an operation status of one or more components of at least onevehicle among the plurality of vehicles.
 6. The central managementsystem of claim 1, wherein the feedback interface further comprises animage or a metric for one or more components of at least one vehicleamong the plurality of vehicles.
 7. The central management system ofclaim 6, wherein the image of the at least one vehicle is a top-view ofthe at least one vehicle.
 8. The central management system of claim 1,wherein the feedback interface activates or deactivates one or moreinterfaces based on the control command being executed by the controldevice of each of the plurality of vehicles.
 9. The central managementsystem of claim 1, wherein the feedback interface configures one or moreinterfaces to blink based on a latency of transmitting the controlcommand.
 10. The central management system of claim 1, wherein thefeedback interface further comprises a map view of the plurality ofvehicles.
 11. A method of controlling a feedback device and a controldevice connected to a server, the method comprising: receiving afeedback interface from the server by the feedback device; controlling adisplay to display the feedback interface on the feedback device;receiving a user input on the feedback interface; transmitting a controlcommand corresponding to the user input from the feedback device to thecontrol device of each of a plurality of vehicles; and when the controldevice receives a first control command and a second control commandwithin a preset time duration, and the second control commandcontradicts the first control command, automatically rejecting thesecond control command, by the control device.
 12. The method of claim11, wherein the control device is configured to automatically reject thesecond control command that is received after the first control commandwithin the preset time duration.
 13. The method of claim 12, wherein thefeedback interface activates or deactivates one or more interfaces basedon a heartbeat signal indicating an operating status of one or morecomponents of at least one vehicle of the plurality of vehicles.
 14. Themethod of claim 11, wherein, when the first control command and thesecond control command that are received within the preset time durationcontradict each other, the control device is configured to calculate avicinity of each of the plurality of vehicles, determine whether thereis any object in the vicinity of each of the plurality of vehicles, andautomatically reject the second control command which, when executed,causes a collision between at least one vehicle of the plurality ofvehicles and the object.
 15. The method of claim 11, wherein thefeedback interface activates or deactivates one or more interfaces basedon the control command being executed by the control device of each ofthe plurality of vehicles.
 16. The method of claim 11, wherein thefeedback interface configures one or more interfaces to blink based on alatency of transmitting the control command.
 17. A server comprising: acommunication interface configured to communicate with a control deviceof each of a plurality of vehicles and a feedback device; a memorystoring instructions; and at least one processor configured to: receivecontrol data from the control device of each of the plurality ofvehicles; determine an operation status of each of the plurality ofvehicles based on the control data; generate a feedback interface basedon the operation status of each of the plurality of vehicles; transmitthe feedback interface to the feedback device; based on a timestampbeing received together with each vehicle action from the plurality ofvehicles, re-determine the operation status of each vehicle when thetimestamp becomes outdated, and automatically modify the feedbackinterface based on the re-determined operation status of each vehicle.18. The server of claim 17, wherein the at least one processor isconfigured to execute: a priority module configured to determine apriority of each of the plurality of vehicles to be moved; a pathminimization module configured to minimize a path distance for each ofthe plurality of vehicles from a first location to a second location;and a time scheduling module configured to obtain a schedule deliverytime for each of the plurality of vehicles and determine a schedule timeto be deployed for each of the plurality of vehicles.
 19. The server ofclaim 17, wherein the control data comprises at least one of vehicledata, positioning data or sensor data.
 20. The server of claim 18,wherein the priority module is further configured to assign the priorityof the plurality of vehicles based on at least one of an urgency of anitem being carried in a vehicle, a duration of the item carried in thevehicle, or a maximum time for delivery assigned to the item or apassenger in the vehicle.